Papers with multi-stage training strategy
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models (2025.findings-emnlp)
Copied to clipboard
Haibo Wang, Zhiyang Xu, Yu Cheng, Shizhe Diao, Yufan Zhou, Yixin Cao, Qifan Wang, Weifeng Ge, Lifu Huang
| Challenge: | Video Large Language Models (VLMs) have been praised for their performance in coarse-grained video understanding but still face ineffective temporal grounding and inadequate timestamp representations. |
| Approach: | They propose a novel Video-LLM that senses and reasoned over specific video moments with fine-grained temporal precision. |
| Outcome: | The proposed model surpasses existing models in fine-grained video understanding tasks and exhibits strong potential as a general video understanding assistant. |
SEAM: Bridging the Temporal-Semantic Granularity Gap for LLM-based Speech Recognition (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing duration-based methods generate embeddings at fixed rates, creating distributional mismatch with LLM pre-training. |
| Approach: | They propose an encoder-decoder architecture that generates embeddings at variable rates through cross-attention between speech features and text embeddables. |
| Outcome: | The proposed architecture achieves competitive performance on LibriSpeech (2.6%/5.2% WER) and 4.7% WER on TED-LIUM-v2 with a multi-stage training strategy and First Token Guidance. |